Approximating Optimal Policies for Agents with Limited Execution Resources

This paper considers the problem of composing or scheduling several (non-deterministic) behaviors so as to conform to a specified target behavior as well as satisfying constraints imposed by the environment in which the behaviors are to be performed. This problem has already been considered by several works in the literature and applied to areas such as web service composition, the composition of robot behaviors and co-ordination of distributed devices. We develop a sound and complete algorithm for determining such a composition which has a number of significant advantages over previous proposals: a) our algorithm is different from previous proposals which resort to dynamic logic or simulation relations, b) we realized an implementation in Java as opposed to other approaches for which there are no known implementations, c) our algorithm determines all possible schedulers at once, and d) we can use our framework to define a notion of approximation when the target behavior cannot be realized. Building and developing re-usable modules is one of the cornerstones of computer science. Furthermore, building on previously established infrastructure has allowed us to construct elaborate and sophisticated structures from skyscrapers and aeroplanes through to the world-wide web. Developing the components that go into these structures is only part of the problem however. Once we have them in place, we need to develop methods for piecing them together so as to achieve the desired outcome. In this paper we consider the problem of composing behaviors. This problem has already attracted some attention in the recent literature [Berardi et al., 2008; Calvanese et al., 2008; de Giacomo and Sardina, 2007; Sardina and de Giacomo, 2007; Sardina et al., 2008; Sardina and de Giacomo, 2008; Berardi et al., 2006b; 2006a] with several proposals being put forward. More precisely, we consider the problem of composing or scheduling several (non-deterministic) behaviors so as to conform to a specified (deterministic) target behavior as well as satisfying constraints imposed by the environment in which the behaviors are to be performed. These behaviors are abstractions that can represent a variety of mechanisms such as programs, robot actions, capabilities of software agents or physical devices, etc. As such, solutions to this problem have a wide field of applicability from composing web services [Berardi et al., 2008] through to co-ordinating multiple robots or software agents [de Giacomo and Sardina, 2007; Sardina and de Giacomo, 2007; Sardina et al., 2008; Sardina and de Giacomo, 2008]. The closest work to this paper is that of Sardina et al. [2008] which proposes a regression based technique to solving this problemwhere we present a progression based technique here and briefly consider the possibility of approximating the target behaviour. For example, consider an urban search and rescue setting with three types of robots. Scout robots can search for victims and report their location. Diagnosis robots can assess victims and determine whether their condition requires special transportation. If not, this robot can guide victims to safety. Rescue robots can carry immobile victims to safety. We will elaborate this example using our framework in this paper. This paper provides four main contributions that improve on previous approaches to this problem: 1) we provide a sound and complete algorithm for solving the behavior composition problem which works in the way of a forward search—this is in contrast to the proposal of Sardina et al. [2008] which can be seen as a backward search; 2) we have realized an implementation of our algorithm in Java which is the first known implementation of a solution to this problem; 3) our algorithm determines all possible schedulers for the target behavior (as can the proposals in [Berardi et al., 2008; Sardina et al., 2008]); and, 4) our approach allows the definition of approximate solutions to the behavior composition problem which can be used when the target behavior cannot be realized by the available behaviors in the given environment.